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ANALYZING THE EFFECTS 

OF THE WEAPON SYSTEMS ACQUISITION REFORM ACT 


ABSTRACT 


The purpose of this project is to analyze the effects of Title II Weapon Systems 
Acquisition Reform Act (WSARA) of 2009 on the Program Executive Office (PEO) 
Combat Support and Combat Service Support (CS&CSS). Our methodology includes a 
literature review as well as interviews with program office stakeholders. We review 
collected data to establish a correlation between WSARA implementation and how 
stakeholders perceive the application of these regulations through the following: 

• Identify the intended effect of Title II WSARA regulations on the 
acquisition process concerning cost, schedule, and performance. 

• Determine the extent to which enacted Title II WSARA has been applied 
within PEO CS&CSS programs. 

• Identify gaps between the intent and application of Title II WSARA 
within PEO CS&CSS, offering recommendations for further research. 

The analysis shows that correlations exist between WSARA implementation and 
improved cost and schedule performance. 
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I. INTRODUCTION 

A. ACQUISITION PROCESS AND THE PERCEPTIONS OF REFORM 

Defense spending currently accounts for over 13% of U.S. government outlays, at 
an anticipated dollar amount of $800 billion for fiscal year (FY) 2014. Acquisitions, or 
procurement (these terms are used interchangeably), spending is approximately 17.5% of 
defense outlays, or nearly $140 billion annually (projected for FY2014). These numbers 
give an order of magnitude to the funds committed annually to create capabilities within 
the U.S. Department of Defense (Government Spending Breakdown, n.d.). 

Concern over the acquisition process appears warranted when considering the 
amount of taxpayer dollars funding this process. The Weapon Systems Acquisition 
Reform Act (WSARA) of 2009 is one of many regulatory reforms aimed at improving 
the complex acquisition process. The following sections present a brief background on 
the acquisition process (system) and historical efforts at reforming this process leading up 
to the implementation of the WSARA in 2009. 

B. THE ACQUISITION SYSTEM “BIG A” 

The acquisition system is composed of the requirements identification process, 
the budgeting and funding process, and the actual process of managing the purchase or 
manufacture of the product. In defense terminology, these three areas are as follows: 

• The Joint Capabilities Integration and Development System (JCIDS): 

This system is responsible for determining requirements and reducing 
program redundancy across military service branches. 

• The Planning, Programmin g , Budgeting, and Execution System 
(PPBES): This system is responsible for forecasting funding resources 
and managing the execution of congressionally allocated and appropriated 
funds. 

• The defense acquisition system (DAS): This is the actual process of 
incorporating user requirements, determining whether to make or buy a 
product to meet those requirements, and the method spanning from this 
requirement identification through the entire life cycle of the resulting 
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product. This is also the focal point for acquisition reform efforts 
(Schwartz, 2013). 

When meshed together, these three areas contribute to the entire acquisition 
process (“Big A”), as depicted in Figure 1. 



Figure 1. The Acquisition System (from Schwartz, 2013) 


C. THE PLANNING, PROGRAMMING, BUDGETING, AND EXECUTION 
SYSTEM 

“A number of issues related to planning and budgeting for national defense confound 
DOD and congressional decision makers annually. Among these are how to perform 
effective and competent threat assessment and the consequences of doing this job well or 
poorly” (Jones & McCaffery, 2008, p. 88). If there were no need to consider fiscal 
constraints when outfitting U.S. military forces, there would be no need for the PPBES. 
However, in an environment of constraints, the PPBES is the U.S. government’s method 
of accounting for, planning, programming, and budgeting for products and systems to 
meet the needs of warfighters and monitor the execution of funds to ensure appropriate 
fund allocation and expenditures. The PPBES is composed of four steps: 
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1. Planning: During the planning stage, the needs of combatant commands 
are analyzed and the findings are published in the Joint Programming 
Guidance document, which guides the DOD components’ efforts to 
propose acquisition programs. 

2. Programming: During the programming stage, proposed programs are 
fleshed out, and the Program Objective Memorandum, a document that 
outlines the anticipated missions and objectives of the proposed weapon 
system and anticipated budget requirements, is submitted to propose these 
programs. The memoranda are reviewed and, as deemed appropriate, 
integrated into an overall defense program. 

3 Budgeting: Budgeting occurs concurrently with the programming stage. 
Proposed budgets are reviewed in a different manner than proposed 
programs. Upon completion of a program decision or as a result of a 
budget review, program budget decisions are issued. 

4. Execution: Execution occurs simultaneously with the program and budget 
reviews. During execution, programs are evaluated and measured against 
established performance metrics, including the rates of funding obligation 
and expenditures. (Schwartz, 2013) 

Much of acquisition reform results from discrepancies in what occurred during 
the planning and programming phases when compared to actual budget execution within 
acquisition programs. 

D. THE ACQUISITION PROCESS “LITTLE A” 

The U.S. Department of Defense (DOD) is by far the largest and most complex 

business organization in the world. It operates more than 5,400 installations 

worldwide and executes more than 15 million contracts per year. It also develops 

and produces the most sought-after weapons and equipment in the free world. 

(Fox, 2011, p. 1) 

The DOD is complex, and the acquisition process is one of the most complex 
processes existing in the DOD. Figure 2 portrays the acquisition process, its phases and 
milestones. While the figure seems simple, the workings of the process are complex. 
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User Needs 


Technology Opportunities & Resources 


• The Materiel Development Decision precedes 
entry into any phase of the acquisition 
management system 

• Entrance criteria met before entering phase 

• Evolutionary Acquisition or Single Step to 
Full Capability 
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Figure 2. Acquisition Process (from DAU, n.d.a) 


The Material Development Decision initiates the Material Solution Analysis 
(MSA) phase of the acquisition process after a user need is identified. The goal of this 
phase is to determine the method (make or buy) required to meet end-user needs. This 
phase includes the Analysis of Alternatives, where acquisition officials consider any 
capability, including commercial off-the-shelf, which can satisfy the requirement with the 
greatest benefit regarding cost, schedule, and performance. Emphasis during this phase is 
on adequately identifying user needs so that there are no requirement misunderstandings 
in the early stages of program consideration. The program moves to Milestone A with 
the creation of the Technology Development Strategy, as it postures for progression into 
the Technology Development (TD) phase. The Milestone Decision Authority (MDA) 
evaluates the program for advancement into the TD phase (DOD, 2013b). 

The goal of the TD phase is to produce technologies mature and capable enough 
to support program requirements. The acquisition strategy—the framework for the entire 
program acquisition process—is developed during this phase and includes developing the 
system’s cost, schedule, and performance criteria. Product prototypes are developed to 
demonstrate initial capabilities required to move into the following acquisition process phase. 
Requests for proposals are disseminated, and contractors begin to compete for the contract 
award. The program manager (PM) develops the preliminary design review, which 
establishes baselines for human, software, and hardware support systems. At Milestone B, 
the MDA determines whether to allow a program to advance into the Engineering and 
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Manufacturing Development (EMD) phase based on teehnological capabilities demonstrated 
and teehnologieal risk reduced during the TD phase (DOD, 2013b). 

A program advances into the EMD phase if the technology has matured, the MDA 
has approved the acquisition strategy, and funding is fully approved. The intent of the 
EMD phase is to demonstrate sound manufacturing, logistical, and end-product 
capabilities. Collaboration with end users during the prior two acquisition phases proves 
imperative because in those phases, key performance parameters (KPPs) are identified as 
the baseline by which the product will be analyzed. Significant testing occurs during this 
phase, and products must demonstrate their capabilities in a relevant developmental 
environment. The EMD phase incorporates all manufacturing, industrial, logistical, and 
human systems into the program, and these systems must demonstrate their capability 
prior to the next milestone decision. When the program has demonstrated these 
capabilities and successfully completed developmental test and evaluation (DT&E), the 
MDA conducts the Milestone C and low-rate initial production (ERIP) analysis, which 
determines whether the program advances to the Production and Deployment phase 
(DOD, 2013b). 

The DOD 5000.02 states that “the purpose of the Production and Deployment 
phase is to achieve an operational capability that satisfies mission needs” (DOD, 2013b, 
p. 27). Operational test and evaluation tests the product or system in an operational 
environment (OE), incorporating end users and support systems equivalent to the OE. 
After the system demonstrates capacity to meet operational needs and meets other 
Milestone C criteria mentioned in DOD Instruction (DODI) 5000.02 Enclosure 9, the 
ERIP decision allows initial production to begin. Key to this phase is the rectification of 
any discrepancies identified during the previous phase, which ensures that issues 
identified during T&E do not persist in the ERIP. Eurthermore, as manufacturing, 
logistical, and industrial systems prove their capability to produce the initial low rates of 
production, these systems are further assessed to determine their capacity to move into 
full-rate production (ERP). ERP leads into the full deployment of the product or system 
to the intended end user (DOD, 2013b). 
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The final phase of the acquisition process is the Operations and Support (OS) 
phase. Prior to entering the OS phase, a program must demonstrate that it has a life-cycle 
sustainment plan (LCSP), among other items. The LCSP is part of the program from the 
beginning and develops in conjunction with the maturing program, ensuring that support 
responsibilities are adequately defined and that trained personnel, equipment, and 
infrastructure fulfilling support and maintenance roles are present. This phase spans the 
remaining life cycle of the program, from FRP to the end of the product or system’s life. 
The program office continues to monitor the product or system’s performance data, 
determining whether adjustments must be made to units in production (DOD, 2013b). 

E. THE JOINT READINESS OVERSIGHT COUNCIL AND JCIDS 

What drives a requirement? Jones and McCaffery (2008) may have stated it best 
when they said, “Threats to national security, perceived or actual, and political priorities 
drive the amount of defense funding requested and appropriated for weapons acquisition” 
(p. 83). The Chairman of the Joint Chiefs of Staff leads the Joint Readiness Oversight 
Council (JROC) through the JCIDS process and the JROC’s integration into the 
acquisition process It is imperative that the JROC be involved with the acquisition 
process from the moment that an end-user requirement is identified. This involvement 
ensures that programs do not suffer from redundancies if a capability exists in a separate 
military branch, allowing for more efficient allocation of procurement funds. “JCIDS 
provides a transparent process that allows the JROC to balance joint equities and make 
informed decisions on validation and prioritization of capability requirements” (DAU, 
n.d.b, p. 1). The JROC and JCIDS fulfill many key roles within the acquisition process, 
as highlighted in the purple portions of Figure 3. 
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ContinuousTedii>ol 09 y Development and Matur^ion 


Figure 3. JCIDS Review Process (from DOD, 2013b) 

DOD components and services conduct capabilities-based assessments that lead 
to a determination of whether a capability gap exists. JCIDS plays a key role by 
increasing collaboration among services. When a capability gap is determined, an initial 
capabilities document (ICD) is generated if non-material solutions have been exhausted. 
Non-material solutions are analyzed on the basis of doctrine, organization, training, 
materiel, leadership, personnel, facilities, and policy (DOTMLPF-P). The JROC assists 
with determining whether the requirement can be met by changing an aspect of 
DOTMLPF-P; otherwise, an ICD is developed. The ICD is the first key document that 
JCIDS contributes to the acquisition system. This document feeds into the MSA and the 
Concept Exploration phase (Chairman of the Joint Chiefs of Staff [CJCS], 2012). 

JCIDS and JROC involvement in the acquisition process remains imperative to 
meeting user needs. JCIDS develops capability development documents, which verify 
that the developing product, system, and technology are capable of moving into the next 
phase of the acquisition process. The JROC regularly interfaces with acquisition’s 
representatives and combatant units, verifying KPPs and ensuring that the program is on 
track with the needs of joint forces and cost, schedule, and performance parameters. 
JCIDS is also responsible for developing the capabilities production documents, which 
indicate that the program’s producability and logistical and infrastructural support 
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systems are mature and capable enough for the program to exit one phase and enter the 
next (DOD, 2013b). 

This complicated process, composed of multiple organizations and many 
stakeholders, has seen many reformation initiatives and regulations over the years. 
Intended benefits of reform varied from a more streamlined process, higher end-user 
satisfaction, and control of cost and schedule overruns. Prior to understanding why 
reform has occurred and whether reform actions have been successful, one must 
understand the process, its complexities, and the differing incentives driving stakeholders 
in and around the acquisition process. 

F. PERCEPTIONS REGARDING ACQUISITION IMPLEMENTATION 

Acquisition reform dates back to as early as 1862 when President Abraham 
Lincoln requested the resignation of the secretary of war, Simon Cameron, due to 
contracting “corruption and mismanagment” (Schwartz, 2013, p. 12). Other acquisition 
reforms continued in the decades that followed, including in 1958, when the Defense 
Reorganization Act further restructured the acquisition process. Modem acquisition 
reform initiatives date back to the early 1990s and encompass a list of 63 initiatives 
(Hanks, Axelband, Lindsay, Malik, & Steele, 2005). More recent efforts led to overhauls 
of major defense acquisition policy, including the acquisition operations manual DODI 
5000 series. However, despite many efforts to reform the acquisition system, multiple 
programs continued to fail at meeting cost and schedule requirements (see Figure 4). 
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Fiscal Year 


2003 

2007 

2008 

Portfolio size 

Number of programs 

77 

95 

96 

Total planned commitments 

$1.2 trillion 

$1.6 trillion 

$1.6 trillion 

Commitments outstanding 

$724.2 billton 

$875.2 billion 

$786.3 billion 

Portfolio Indicators 

Change to total RDT&E costs from first estimate 

37 percent 

40 percent 

42 percent 

Change to total acquisition cost from first 
estimate 

19 percent 

26 percent 

25 percent 

Total acquisition cost growth 

$183 billion 

$301.3 billion" 

$296.4 billion 

Share of programs with 25 percent increase in 
program acquisition unit cost growth 

41 percent 

44 percent 

42 percent 

Average schedule delay in delivering initial 
capabilities 

18 months 

21 months 

22 months 


Source: GAO analysis of DoD data. 


Notes; Data was obtained from DoD’s Selected Acquisition Reports (SAR) (dated December 
2002, 2006, and 2007). In a few cases, data were obtained directly from program offices. The 
number of programs reflects the programs with SARs; however, in our analysis we have broken 
a few SAR programs into smaller elements or programs. Not all programs had comparable cost 
and schedule data and these programs were excluded from the analysis where appropriate. 
Portfolio performance data do not include costs of developing Missile Defense Agency 
elements or the Defense Integrated Military Human Resources System (DIMHRS) program. 
The total acquisition cost growth for the 2007 portfolio was $295 billion in 2008 constant dollars 
(GAO-09-663T. p. 0). 

Figure 4. Analysis of DOD MDAP Portfolios (from Francis, 2009) 

These failures to meet requirements led to another round of acquisition reforms, 
including the WSARA of 2009 and the Better Buying Power initiatives. These reforms 
again intended to reduce cost and schedule overruns while improving performance. 

G. CONCLUSION 

The number of initiatives spanning approximately 23 years of acquisitions—63 
initiatives in total—is one of the key points from the RAND study conducted by Hanks et 
al. (2005). These initiatives are often mere policy documents generated as a new leader 
assumes position within the purview of the acquisition process and rarely find their way 
into actual acquisition regulation. Another issue plaguing acquisitions and reform efforts 
is discussed in a more recent study conducted by the Government Accountability Office 
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(Francis, 2013). This study concluded that the incentives driving multiple stakeholders 
involved with the acquisition process differ vastly and continue to frustrate the 
acquisition system. Our research analyzes incentives and how current reward and 
evaluation systems may not be aligned with regulation implementations (such as the 
WSARA of 2009) and create a disconnect between regulation and activities in an 
acquisition organization. 
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II. LITERATURE REVIEW AND WSARA BACKGROUND 


In Chapter I, we established the foundation of this research. In this chapter, we 
provide a literature review surrounding the WSARA. This review covers the following: 

• establishment and implementation of the WSARA, 

• WSARA implementation and program and product successes, 

• desired effects are occurring from the WSARA, and 

• cost, schedule, and performance and contract metrics. 

To capture the complex timeline of reform initiatives, the paragraphs in this 
chapter are organized by the project’s objectives; resources are described in 
chronological order. 

A. LITERATURE SELECTION 

The literature surrounding acquisition reform helps to illuminate and determine 
the project objectives. To determine whether there is a correlation between WSARA 
implementation and program and product success, we first examine the need for 
establishment of the WSARA and the key aspects of acquisition reform. Next, to 
determine whether WSARA legislation is having the desired effects, we address current 
legislative debate and articles. Finally, to assess the effects of WSARA legislation on 
cost, schedule, performance, and contract metrics, we look at the WSARA’s intended 
effects, perceived effects, and actual effects. We also show how effects-based 
management theory can be tied to WSARA implementation and describe why there is a 
need for an accurate assessment on the measures of performance related to the WSARA. 

We selected the literature because there is not much data on the WSARA or its 
actual effects. We chose a variety of mediums due to the ambiguity surrounding the 
topic. To see more deeply into the subject matter, we selected journal articles and books 
from experts on acquisition reform. We also selected key GAO reports to illustrate 
ongoing challenges and analysis. We also chose congressional transcripts that cover 
acquisition reform and some debate on the WSARA. And finally, we selected relevant 
news articles to reflect public understanding and interpretation of the legislation and 
acquisition reform issues. 
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1. Establishing a Need for the WSARA 

The WSARA was a necessary piece of legislation. Its intent was “to improve the 
organization and procedures of the Department of Defense for the acquisition of major 
weapon systems, and for other purposes” (Weapon Systems Acquisition Reform Act 
[WSARA] of 2009). Decades of acquisition reform and initiatives led up to the 
implementation of the law. Hanks et al. (2005) identified 63 acquisition reform 
initiatives from the 1960s to 2001 that were aimed at streamlining logistics; improving 
the industrial base; managing total life-cycle system management; and reducing waste, 
fraud, and abuse. 

The Military Reform Caucus (MRC), cofounded in part by Newt Gingrich, was 
originally founded in 1981 (Malishenko, 1987). Malishenko (1987) highlighted 
congressional dynamics in his article with the MRC. It sheds light on early attempts for 
bipartisanship between political parties of the House and Senate in the 1980s during the 
early years of the acquisition reform initiatives. The MRC membership was based on 
interested congressional members and helped lay the foundation for future legislation 
(Malishenko, 1987). 

Thompson (1992) described his frustration with the number of laws and extra 

layers of bureaucracy added to the DOD acquisition processes and stated that “greatly 

increased regulation and oversight [are] needed to fix the system” (p. 729). Furthermore, 

Congress “added an assortment of new laws aimed at further disciplining DOD 

acquisitions to the existing 1,150 feet of legislation and case law governing procurement” 

(Thompson, 1992, p. 729). Thompson (1992) highlights that 

most acquisition experts call for the elimination of the statutes that limit the 
proper exercise of the acquisition officer’s discretion. They all call for a 
substantial reduction in the size and scope of the systems commands, systems 
specifications, evaluation teams, and in the number of criteria the evaluation 
teams consider in the source selection process. Most call for clear command 
channels, limited reporting requirements, and project teams with small high- 
quality staffs. (Thompson, 1992, p. 744) 

Many reform initiatives were established leading up to the WSARA law. Most 
notable is President Reagan’s Packard Commission report from 1986. The Packard 
Commission “concluded that the primary problems with the acquisition process were the 

12 



same ones identified in previous decades (cost growth, schedule delays, performance 
shortfalls)” (Christensen, Stearle, & Vickery, 1999, p. 251). Through their study, 
Christensen et al. (1999) highlighted previous attempts of legislation that were not 
adequate in improving cost performances: “Despite the implementation of more than two 
dozen regulatory and administration initiatives, there has been no substantial 
improvement in the cost performance of defense programs for more than 30 years” 
(p. 252). 

A 2005 RAND report (Hanks et ah, 2005) highlighted acquisition reform 
initiatives that illustrate 63 initiatives from 1966 up to 2001 that were marginally 
successful. Several of the interview quotations from the RAND report emphasized this 
theme: 

• “Acquisition Reform inhibitors: need to change law or just working the 

edges; need to change how funding is done. Example: _ program 

subsidizes everything at _ (Major Subordinate Command), so the 

incentives are to keep selling parts [to that program] and not let them fix 
the parts” (program executive officer-military; p. 121); 

• “The test community is still focused on their reporting requirements rather 
than testing to fix” (PM-military; p. 110). 

In their analysis of organizational transformation, Eide and Allen (2012) 

explained acquisition reform legislation in clear and concise detail. Starting with the 

Packard Commission report of 1986, Eide and Allen (2012) specifically stated. 

Excellence in defense management will not and cannot emerge by legislation or 
directive. Excellence requires the opposite—responsibility and authority placed 
firmly in the hands of those at the working level, who have knowledge and 
enthusiasm for the tasks at hand (p. 102). 

The Goldwater-Nichols Act of 1986 followed the Packard Commission report 
and addressed some of the changes that the Packard Commission suggested, specifically 
“diluted authority for execution” (Eide & Allen, 2012, p. 102). Moving up a decade into 
the 1990s, a more business-minded aspect became apparent in legislation and reform 
initiatives. Eocusing on the acquisition workforce, “The Defense Acquisition Workforce 
Improvement Act (DAWIA) of 1990 addressed the need to improve the quality of the 
acquisition workforce, establishing formal career paths and standards for education and 
training” (Eide & Allen, 2012, p. 102). 
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President Clinton signed two laws focused on acquisition reform during his 
presidency: the Federal Acquisition Streamlining Act (FASA) of 1994 and the Clinger- 
Cohen Act of 1996. To broaden the definition of what a commercial item was, “FASA 
streamlined procurement of commercial items by exempting them from existing laws” 
(Fide & Allen, 2012, p. 102). The Clinger-Cohen Act of 1996 facilitated the way that 
the government acquires information technology by “eliminat[ing] cost accounting 
standards that had discouraged commercial companies from doing business with the 
federal government” (Fide & Allen, 2012, p. 102). 

In 2005, the Defense Acquisition Performance Assessment Project was 
established to provide an assessment of the DOD’s acquisition system. Major findings of 
the project included identification of “excessive oversight and complex acquisition 
processes as cost and schedule drivers, and called for stability of requirements as an 
essential element for an effective acquisition system” (Fide & Allen, 2012, p. 104). 

The WSARA aimed at improving the success of major defense acquisition 
programs (MDAPs). To accomplish this task, the WSARA provided focus on early 
decision variables such as “reliable and independent baseline cost estimates, rigorous 
early developmental testing and systems engineering oversight, and strong gatekeeping to 
prevent programs from proceeding with too much risk of immature technology” (Fide & 
Allen, 2012, p. 106). Along with program success, an implied goal of the WSARA was 
to reduce costs and risk. Increased competition was incorporated in the WSARA to help 
accomplish this goal (Fide & Allen, 2012, p. 106). 

The WSARA’s key provisions altered the DOD’s acquisition bureaucracy in 
addition to process systems and senior management oversight changes (Berteau, 
Hofbauer, & Sanok, 2010, p. 4). The WSARA also legislated important changes on 
acquisition processes. According to Berteau et ah, WSARA 

• increased competition throughout the acquisition process, 

• improved requirement formulation processes, 

• improved cost-estimation processes, 

• a more stringent set of regulations on organizational conflicts of interest. 


revised Milestone A and Milestone B certification processes, 
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• revised processes for reporting critical cost growth, and 

• increased congressional oversight through heightened reporting 
requirements 

The WSARA created four senior-level Office of the Secretary of Defense 
positions with military-service-level equivalents for the DOD (Berteau et ah, 2010; 
WSARA of 2009): 

• Director of Cost Assessment & Program Evaluation (DCAPE) 

• Director, Development Test & Evaluation (DT&E) 

• Director, Systems Engineering (SE) 

• Director for Performance Assessments and Root Cause Analyses 
(PARCA) 

The primary responsibilities of these offices are illustrated in Eigure 5. 


Table 2: Key Responsibilities of Offices Established as a Result of the Reform Act 


Office 

Primary responsibilities 

Systems Engineering 


serves as principal advisor to the Secretary of Defense and the Under Secretary of [>efense 

for Acquisition, Technology and Logistics on systems engineering activities in the department 

develops systems engineering and development planning guidance for DOD 

reviews and approves major defense acquisition program systems engineering plans 

monitors major defense acquisition program systems engirteering activities 

reports to Congress annually on systems engineering organization, capabilities, and activities 

Developmental Test and 
Evaluation 


serves as principal advisor to the Secretary of Defense and the Under Secretary of Defense 
for Acquisition, Technology and Logistics on developmental test and evaluation activities 
develops developmental test and evaluation guidance for DOD 

reviews and approves major defense acquisition program developmental test and evaluation 
plans 

monitors and reviews acquisition program developmental test and evaluation activities of 
major defense acquisition programs 

reports to Congress annually on developmental test and evaluation organization, capabilities, 
and activities 

Cost Assessment and Program 
Evaluation 


serves as principal advisor to the Secretary of Defense and other senior officials on matters 
related to cost analysis and the planning and programming phases of the planning, 
programming, budgeting, and execution system 

develops independent cost estimates for major defense acquisition programs prior to major 
milestone decisions and updates independent cost estimates when programs have exceeded 
critical cost thresholds, known as Nunn McCurdy breaches 

reviews existing systems and methods for tracking and assessing operation and support 
costs on major defense acquisition programs 

develops analysis of alternative study guidance for major defense acquisition programs 
approves the analysis of alternatives study plan for each major defense acquisition program 

Performance Assessments and 
Root Cause Analyses 


assesses major acquisition program performance through independent analyses and through 
the Defense Acquisition Executive Summary process 

identifies the root causes of cost growth and other problems on programs that experience a 
critical Nunn McCurdy cost breach 


Souicb: GAO AnalyM of Weapon Syslenv Acq w aon Reform Act of 2009. 


Eigure 5. WSARA Key Office Responsibilities (from Sullivan, 2012, p. 5) 
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2 . 


WSARA Implementation and Correlation to Program and Product 
Success 


Finding the relationship between a legislation’s intended effects and its actual 
results on the group to be legislated is challenging. Legislation rarely has a direct visual 
result on the DOD’s bottom line but rather creates a waterfall of changes and secondary 
effects. A correlation between two variables can exist, but causation is a different matter. 
In our research of the WSARA law and its provisions, we make an assumption that the 
WSARA is the initiator of bottom-line change, if not the catalyst for changes that occur. . 

The bottom line in the corporate world is profit. For a governmental entity like 
the DOD, the acquisition bottom line is measured in cost, schedule, and performance. 
There is a correlation between other acquisition initiatives and cost, schedule, and 
performance metrics. Biery (1992), along with some other researchers, argued that the 
DOD was doing well during the 1990s, positing that “if meeting cost and schedule 
objectives measures an institution’s managerial efficiency, then the DOD is more 
efficient than the typical commercial business or government organization” (p. 644). 
Biery (1992) also illustrated that the end of each decade has a decreasing trend of cost 
growth and compound annual growth rate. 

Countering Biery’s (1992) argument, Christensen et al. (1999) conducted a cost 
overrun analysis on 269 contracts that were four years before and four years after the 
implementation of the Packard Commission recommendations. The assumption was that 
there would be a clear drop in cost overruns. However, Christensen et al.’s (1999) 
results, as seen in Figure 6, “show that the Packard Commission’s recommendations did 
not improve the cost performance of defense acquisition contracts” (p. 256). 
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Cost Performance 



Contracts 


■ Before 

■ After 


Figure 6. Graph from Cost Overrun Analysis on Packard Commission Reforms 

(from Christensen et ah, 1999) 


The WSARA’s implementation has had success, as illustrated in the third GAO 
report on the WSARA, authored by Sullivan (2012). According to the report, the 
“GAO’s analysis of 11 weapon acquisition programs showed the Reform Act has 
reinforced early attention to requirements, cost and schedule estimates, testing, and 
reliability” (Sullivan, 2012, p. 1). This is reinforced by GAO Report No. GAO-14-145T, 
authored by Francis (2013); Francis indicated that “to the extent reforms like the Weapon 
Systems Acquisition Reform Act and DOD’s Better Buying Power initiatives are being 
implemented, they are having a positive effect on individual programs” (p. 5). 

3. Desired Effects From the WSARA 

The Department of Defense’s Rapid Acquisition Process: Is It a Model for 
Improving Acquisition, the first hearing before the Panel on Defense Acquisition Reform 
of the House Armed Services Committee on June 3, 2009, highlighted ongoing 
challenges associated with the DAS. During the hearing, Gordon England, a former 
deputy secretary of defense and president of the company E6 Partners, spoke on the 
complexity of the DAS: 
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Counterintuitively, that means you want to give managers more flexibility. 

The more complex the system, the more flexibility you need, managers 
need. The trend is always the other way. That is it gets more complex, we 
add layers of bureaucracy and regulation and control and that makes it 
almost impossible to run very complex programs. So the system today is 
way overburdened. It is over-burdened by the Department, it is 
overburdened by the Congress. As it becomes a more complex system, we 
need to simplify it, otherwise managers won't be able to operate. {The 
Department of Defense’s Rapid Acquisition Process, 2009, p. 6) 

Representative Jim Cooper from Tennessee mentioned additional complexity 
added to the acquisition process from the WSARA but unfortunately suggested that no 
one read the law, stating, “I am worried about the Tower of Babel effect when we create 
a system that is so complex that nobody can understand it. We were just joking prior to 
the hearing that how many people actually read the weapons acquisition bill that we just 
passed. Nobody” {The Department of Defense’s Rapid Acquisition Process, 2009, p. 17). 
Robert Andrews, the chairman of the panel at the time, addressed the lack of coordination 
between requirements, budgeting, and procurement {The Department of Defense’s Rapid 
Acquisition Process, 2009, p. 10). Luckily, the WSARA addressed some of Andrews’ 
concerns. 

Programs are just beginning to implement the provisions of the WSARA 
(Sullivan, 2012). The report points to little to no improvement in acquisition areas. 
However, according to a 2013 report by the Government Business Council, there has 
been improvement in personal accountability (as shown in Figure 7). 
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Figure 7. Acquisition Areas of Improvement, Affirmative Answers 

(from Government Business Council, 2013, p. 21) 


4. Conclusion 

This chapter introduced past and current literature on the establishment and 
implementation of the WSARA; correlation between WSARA implementation and 
program and product successes; intended effects of the WSARA; cost, schedule, and 
performance and contract metrics; and effects-based management. The next chapter 
describes research methodology, our data collection process, a description of the Program 
Executive Office for Combat Support and Combat Service Support (PEO CS&CSS), a 
list of our survey questions, and the analysis we conducted. 

The literature covering this topic has several agreements. Eirst, consistent 
acquisition reform is needed as requirements are continuously generated from threat- 
based analysis within budgetary constraints; refinement and reform of the acquisition 
system is needed. Second, acquisition reform is ongoing. As mentioned, threats and the 
operating environment are always changing. Therefore, timely and accurate reform is 
needed in legislation and policies to develop the right solution at the right time. Third, 
flexibility is needed in a complex system. The ability to provide resources to the right 
people to execute solutions in a complex system is paramount. Eifth, more accurate 
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metrics are needed on cost, performance, and schedule to provide senior leadership with 
the right information to make decisions. 

The literature covering this topic has a couple of disagreements. First, several of 
the authors at different points in time (both current and past) have disagreed on whether 
acquisition reform, the WSARA included, is doing a good job. Hindsight is 20/20—it is 
easier to measure reforms 10 years after they have been implemented and the entire 
operating environment has changed. Second, there is disagreement on how success is 
measured. Managers at the program and product levels have a different interpretation of 
which metrics should be used versus those at Congress or in the private sector. 

Our project covers a specific case study. We focus on the PEO CS&CSS and its 
products. By focusing on a single office and seeing how the WSARA is being 
implemented there, we can provide a more accurate sample rather than providing a 
blanket population or randomly selected programs. We can also identify successes and 
challenges more accurately. The lessons learned at a specific program office could then 
be utilized at other offices as the personnel see fit. Furthermore, we use an analysis on 
effects-based management and how it could be implemented into different levels of 
management to yield measurable results. 
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III. METHODOLOGY 


A. INTRODUCTION 

In this chapter, we address the organizational structure of the PEO CS&CSS and 
then explain our interview and quantitative data methodologies. We generated interview 
questions and research efforts utilizing the primary and secondary research questions for 
this project. The project questions are as follows: 

• Overarching project question: How are WSARA Title II policies being 
applied in MDAPs? 

• Why was the WSARA needed, or what were the precursors that led to the 
WSARA? 

• What was the intended effect of the WSARA? 

• How effective has the implementation of the WSARA been in the PEO 
CS&CSS? 

• Are there gaps between the intent and application of Title II of the 
WSARA within a major PEO? If so, why? 

Our intent with these questions was to determine whether acquisition reforms, 
such as those included in Title II of the WSARA of 2009, have their desired effect in 
major acquisition programs. 

B. PEO CS&CSS LOCATION AND ORGANIZATIONAL STRUCTURE 

The PEO CS&CSS is located within the Tank and Automotive Command 
(TACOM) headquarters perimeter in Warren, MI, and is one of four PEOs within the 
TACOM structure. The PEO CS&CSS follows the typical major acquisition 
organization: the program executive officer and his or her deputies orchestrate the 
operations of the PMs (civilian or military members in Grade 06 [colonel] or the civilian 
equivalent) and product managers (PdMs; civilian or military members in Grade 05 
[lieutenant colonel] or the civilian equivalent) and their various assistants and staff. Eor 
the PEO CS&CSS, the PM level is broken down into the following categories: 

• Eorce protection: This category includes bridging, combat engineering, 
force sustainment, petroleum and water, sets, kits, outfits and tools, test 
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measurement and diagnostic equipment, and contingency basing 
infrastructure; 

• The Joint Program Office for the Joint Light Tactical Vehicle (JLTV): 
This category includes the JLTV, the joint Army-Marine acquisition effort 
currently in the EMD phase of the acquisition life cycle; 

• Mine-resistant, ambush-protected vehicles (MRAP): This category 
includes MRAP-ATV systems and joint logistics; 

• Mobile electric power: This category includes small, medium, large, and 
battery-powered electrical power sources; and 

• Transportation systems: This category is composed of personnel and 
logistics transportation systems, including medium tactical vehicles, 
armored security vehicles, heavy tactical vehicles, allied tactical 
vehicles, and Army watercraft systems (PEO CS&CSS, n.d.). 

C. INTERVIEWEE SELECTION 

In late October 2013, a list of Army-sponsored research projects was distributed 
through the Naval Postgraduate School’s Graduate School of Business and Public Policy 
channels. Among the projects on the list was a research project sponsored by the PEO 
CS&CSS stating a request to research acquisition reform and determine whether reforms 
are implemented and the intended effects are being achieved. In this project, we focused 
the question concerning acquisition reform to include the WSARA of 2009, specifically 
Title II. Title II of the WSARA was determined to be the portion of the legislation that 
was most readily translatable into interview and research questions supporting the greater 
request issued by the PEO CS&CSS. 

We began correspondence with the PEO CS&CSS-designated point of contact to 
determine interviewee availability. We requested individuals operating at the PM and 
PdM levels as the primary interviewees, with the understanding that these personnel are 
ultimately responsible for managing the cost, schedule, performance, and risk of PEO 
CS&CSS programs. Cost, schedule, performance, and risk are the focal points of the 
Title II WSARA reforms. 
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D. INTERVIEW METHOD 


We conducted interviews on-site with the PEO CS&CSS in Warren, ML We 
utilized this methodology in order to achieve accurate and spontaneous responses from 
interviewees. Rather than utilizing an e-mailed survey or questionnaire, which potentially 
would allow interviewees to manipulate or research a best answer to a question, face-to- 
face interviews afforded honest and candid responses in situ. Additionally, conducting 
interviews on-site allowed us to develop perceptions concerning the PEO CS&CSS and 
its priorities that could not be gleaned otherwise. 

E. INTERVIEW QUESTION DEVELOPMENT 

We developed the interview questions based on Title II of the WSARA as well as 
expectancy management concepts. Eirst, we converted each subsection of the Title II 
WSARA legislation into the form of a question. The answers to the question would serve 
as a means for qualifying the effects of the WSARA at the PEO level. We then 
categorized the interview questions based on the legislation in order to nest them for 
analysis of overarching project questions. Second, we developed questions that focused 
on the interviewee’s interpretation of the effectiveness of the WSARA, the incumbent 
challenges as well as successes. 

F. DEFENSE ACQUISITION MANAGEMENT INFORMATION 

RETRIEVAL SYSTEM DATA 

The Defense Acquisition Management Information Retrieval System (DAMIR) 
compiles multiple sources from MDAP information systems as well as major automated 
information systems. According to its website, DAMIR is “the authoritative source for 
Selected Acquisition Reports (SAR), SAR Baseline, Acquisition Program Baselines 
(APB), and Assessments” (DAMIR, 2014). 

We gathered data from DAMIR with the intent of conducting a high-level 
quantitative analysis on the data. We were looking for a correlation between program 
bottom-line numbers and when the WSARA was implemented. We chose SARs for all 
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Army programs present in the DAMIR system from 2005 to 2012. According to the 
press release (DOD, 2013a): 


SARs summarize the latest estimates of cost, schedule and performance 
status. These reports are prepared annually in conjunction with the 
submission of the President’s Budget. ...The total program cost estimates 
provided in the SARs include research and development, procurement, 
military construction, and acquisition-related operation and maintenance. 

Total program costs reflect actual costs to date as well as future 
anticipated costs. All estimates are shown in fully inflated then-year 
dollars. 

The SAR categories that we collected categorized the Army programs by the 
following: 


• current estimate, 

• contractor, 

• unit cost by percentage change, 

• sunk funding, 

• cost variance, and 

• schedule. 

The sample data had 17 programs. There was no Army SAR data available for 
calendar year (CY) 2008. We then placed the data into a pivot table and gave the data 
sparklines to identify and illustrate trends. The dependent variable was the report 
categories’ cost or given numerical value. The independent variable was the SAR report 
calendar date. 

G. CONCLUSION 

The interviews and cost data were used to gain a qualitative and quantitative 
perspective on the topic. The interviews provided a focused view of the legislation’s 
impact on a program office. The cost data provided a global view on the legislation’s 
effect on the Army’s MDAP portfolio. 
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IV. DATA AND ANALYSIS 


A. INTRODUCTION 

This chapter addresses the organizational structure of PEO CS&CSS followed by 
an analysis of interview questions driven by the primary and subordinate research 
questions for this project. These questions are listed below: 

1. Overarching project question: How are WSARA Title II policies being 
applied in MDAPs? 

2. Why was the WSARA needed, or what were the precursors that led to the 
WSARA? 

3. What was the intended effect of the WSARA? 

4. How effective has the implementation of the WSARA been in PEO CS&CSS? 

5. Are there gaps between the intent and application of Title II WSARA within a 
major PEO, why? 

The intent is to determine whether acquisition reforms, such as those included in 
Title II of the WSARA of 2009, have their intended effect in major acquisition programs. 
On-site interviews were conducted to determine the extent that the WSARA is 
understood and implemented in PEO CS&CSS. Additional data were collected through 
analysis of program data obtained from the DAMIR website and other related program 
documents for amplifying information. The interview questions are grouped under the 
thesis questions, followed by our analysis of responses. We omit the first question in this 
chapter but return to it in the conclusions and recommendations chapter. 

B. PEO CS&CSS ORGANIZATIONAL STRUCTURE 

Major acquisition organizations follow a structure where the Program Executive 
Officer (PEO) and his or her deputies orchestrate the operations of the Program Managers 
(PM, civilian or military member in the grade of 06 [colonel] or civilian equivalent). 
Product Managers (PdM, civilian or military member in grade 05 [lieutenant colonel] or 
civilian equivalent), with their various assistants and staff following this structure as 
depicted in Eigure 8. 
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Figure 8. Example PEO Staff Structure 


Eor PEO CS&CSS the PM level is broken down into the following categories: 

• Force Protection: This includes bridging, combat engineering, force 
sustainment, petroleum and water, sets, kits, outfits and tools (SKOT), test 
measurement and diagnostic equipment and contingency basing 
infrastructure. 

• Joint Program Office for the Joint Light Tactical Vehicle (JLTV): 

JETV is the joint Army-Marine acquisition effort currently in the 
Engineering and Manufacturing Development (EMD) phase of the 
acquisition life cycle. 

• Mine Resistant Ambush Protected Vehicles (MRAP): This includes 
MRAP-ATV (MATV) systems and joint logistics. 

• Mobile Electric Power: This category includes small, medium, large, and 
battery-powered electrical power sources. 

• Transportation Systems: This category is comprised of personnel and 
logistics transportation systems including medium tactical vehicles, 
armored security vehicles, heavy tactical vehicles, allied tactical vehicles 
and Army watercraft systems. 

PEO CS&CSS manages over 350 systems in total. Our initial interviews were 
conducted with various representatives within this organizational structure. 

1. Task Allocation 

WSARA legislation called for additional high-level oversight at OSD. The 
interviews indicated that the PEO staff structure was in place to match what the WSARA 


26 



















required at OSD level as shown in Figure 5 of our literature review section. Project and 
product offices had teams or individuals that would handle data calls required by PEO 
staff or OSD level. Additionally, while the organization was structured in accordance 
with regulation, PdMs were capable of adapting their structure based on their program’s 
current phase within the Integrated Defense Acquisition, Technology, and Logistics Life 
Cycle Management System. 

Work tasks are allocated in a hierarchical fashion from PEO to project office to 
product office. At the product office level, tasks are allocated by PdM identified 
priorities. 

C. WSARA QUANTITATIVE IMPACTS 

The intended effect of the WSARA was “to improve the organization and 
procedures of the Department of Defense for the acquisition of major weapon systems, 
and for other purposes” (WSARA of 2009, 2009), with the secondary intent to reduce 
risk and costs. To accomplish this, the legislation focused on changing senior 
management oversight and improving early decision variables in the acquisition system. 

As established in the literature review, the need for the law was clear. Cost 
overruns were ever increasing in frequency and value. There was a recognized need “to 
improve the way weapon systems are acquired” (Sullivan, 2012, p. 4). The intent of the 
law was to limit those overruns and enhance oversight and accountability by adding 
additional layers of oversight and streamlining the monitoring process to “ensure that 
costs are controlled” (Sullivan, 2012, p. 8). 

I. DEFENSE ACQUISITION MANAGEMENT INFORMATION 
RETRIEVAL 

DAMIR compiles multiple sources from MDAP and Major Automated 
Information System (MAIS) information systems. According to its website, DAMIR is 
“the authoritative source for Selected Acquisition Reports (SAR), SAR Baseline, 
Acquisition Program Baselines (APB), and Assessments” (DAMIR, 2013). We pulled 
relevant data from DAMIR. The data retrieved were from the Army’s 2005-2012 SARs 
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for noted MDAPs. These sources of data are relevant as stated by the Department of 
Defense: 

SARs summarize the latest estimates of cost, schedule and performance 
status. These reports are prepared annually in conjunction with the 
submission of the President’s Budget. Subsequent quarterly exception 
reports are required only for those programs experiencing unit cost 
increases of at least 15 percent or schedule delays of at least six months. 

The total program cost estimates provided in the SARs include research 
and development, procurement, military construction, and acquisition- 
related operation and maintenance. Total program costs reflect actual 
costs to date as well as future anticipated costs. All estimates are shown in 
fully inflated then-year dollars. (DOD, 2013a) 

The reports were chosen because they reflect the most relevant data on the 
Army’s MDAPs from 2005 to 2012. Those years were selected to analyze changes 
occurring in the period four years before the WSARA and three years after. The intent of 
analyzing the years before and after the WSARA was implemented to see any changes 
within the sample on cost (which was the WSARA’s secondary objective and also the 
best quantitative measurement for success). There were no Army SARs listed in DAMIR 
forCY2008. 

2. Trends in Cost Data 

We specifically analyzed three quantitative cost measures to see whether there 
were any trends: current estimate total cost; Program Acquisition Unit Cost (PAUC) 
percent change compared to the baseline estimate; and Average Procurement Unit Cost 
(APUC) percent change compared to the baseline estimate. We were looking to see 
whether costs were increasing, decreasing, or varied before and after implementation of 
the WSARA. In addition, we wanted to see whether there was variation of cost changes 
before and after implementation of the WSARA. Below are figures from our quantitative 
analysis that show the trends in available data for the programs based off the current 
estimate, PAUC, and APUC for each program: 
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Figure 9. SAR Current Estimate, Total Cost in Millions ($) 
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-4.38% 
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-8.00% 
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Figure 10. SAR PAUC % Change Against Original Baseline 
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SAR APUC % Change Against Original Estimate Baseline 
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2006 
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2009 

2010 
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2012 


AH-64E New Build 

0.00% 

0.00% 

0.00% 

0.00% 
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-13.78% 
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0.00% 
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-0.01% 
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1 

1 
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3.84% 

6.23% 
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0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

-1.67% 


UH-60M Black Hawk 

9.70% 
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12.88% 

16.48% 

19.02% 
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i — ^ 
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0.00% 

0.00% 

0.00% 
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0.00% 

0.00% 

0.00% 

-7.97% 

-7.45% 

-15.92% 

-10.82% 


Figure 11. SAR APUC % Change Against Original Baseline 


D. CENTRAL TENDENCY 

To understand variation in cost measure, we analyzed three measures of 
variability in central tendency. The intent was to find the measurement that provides the 
best representation of the data. We used the SARs to gather data on the standard 
deviations, averages, and coefficient of variation (CV) for the specified cost element 
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current estimate, PAUC, and APUC % changes). First, averages (means) were used to 
gauge the total average of each program. Second, standard deviation was used to view 
the average deviation of values from the mean. Third, CV was used to see the dispersion 
of data relative to the mean because it allows for analysis of variation between the 
programs that have significant cost differences. 

The data provided by DAMIR had 17 Army MDAP programs listed from 2005 to 
2012. Data taken as “before WSARA” was from 2005 to 2009 SARs. Data taken as 
“after WSARA” was from 2010 to 2012 SARs. For averages, standard deviation, and 
coefficient of variation, the data were taken from each program’s SAR before and after 
the WSARA, respectively. 

Table 1 summarizes the data collected and calculated; the details are discussed in 
the following three paragraphs. Dollar values are in millions. The percent changes for 
PAUC and APUC are the change compared to the original baseline estimate from the 
SAR. 


Table 1. SAR Measures of Central Tendency 


AVERAGES 


Current 

PAUC % 

APUC % 


Estimate 

Change 

Change 

Programs that increased 

11 

5 

5 

Average increase before WSARA 

$10,210.16 

5.16% 

8.69% 

Average increase after WSARA 

$10,370.26 

40.04% 

14.79% 

Average increase net 

$160.09 

34.88% 

6.10% 

Programs that decreased 

6 

12 

12 

Average decrease before WSARA 

$10,552.49 

4.72% 

3.13% 

Average decrease after WSARA 

$6,014.36 

-2.73% 

-6.02% 

Average decrease net 

$(4,538.13) 

-7.45% 

-9.15% 

All Programs average before 
WSARA 

$10,347.09 

4.87% 

4.98% 

All Programs average after 

WSARA 

$8,832.88 

9.85% 

0.10% 

All Programs average net change 

$(1,514.21) 

4.98% 

-4.06% 






STANDARD DEVIATIONS 


Current 

PAUC % 

APUC % 


Estimate 

Change 

Change 

Programs that Increased 

9 

8 

10 

stdev increase before WSARA 

$297 

0.0418 

0.0308 

stdev increase after WSARA 

$1,198 

0.2221 

0.0979 

stdev increase net 

$901 

0.1803 

0.0671 

Programs that decreased 

8 

9 

7 

stdev decrease before WSARA 

$1,572 

0.0953 

0.0963 

stdev decrease after WSARA 

$601 

0.0357 

0.0308 

stdev decrease net 

$(971) 

(0.0596) 

(0.0655) 

All Programs stdev before WSARA 

$1,224 

0.0856 

0.0578 

All Programs stdev after WSARA 

$917 

0.1234 

0.0702 


All Programs stdev net change_$(307)_0.0379_0.0125 


COEFFICIENT OF VARIATION 



Current 

PAUC % 

APUC % 


Estimate 

Change 

Change 

Programs that increased 

10 

1 

10 

CV increase before WSARA 

2.76% 

58.92% 

68.29% 

CV increase after WSARA 

16.40% 

117.74% 

101.69% 

CV increase net 

13.64% 

58.82% 

33.40% 

Programs that decreased 

7 

7 

5 

CV decrease before WSARA 

22.50% 

226.34% 

124.99% 

CV decrease after WSARA 

8.25% 

95.22% 

39.65% 

CV decrease net 

-14.24% 

-131.12% 

-85.34% 

All Programs CV before WSARA 

15.32% 

176.11% 

99.79% 

All Programs CV after WSARA 

13.04% 

106.48% 

81.01% 

All Programs CV net change 

-2.27% 

-69.63% 

-18.78% 


I. SAR Current Estimate 

The first cost estimate was the SAR “current estimate,” which is the most recent 
estimate of the program’s parameters reflecting the President’s Budget proposal as well 
as unforeseen circumstances or unavoidable circumstances that adjusted cost measures. 
(Hagan, 2012, p. B-55). 
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Before WSARA implementation, 11 programs increased in dollar value from the 
current estimate. The average value of the 11 programs before the WSARA was $10.21 
billion. The average cost estimate of the 11 programs after the WSARA was $10.37 
billion, yielding a net increase of $160 million. After WSARA implementation, six 
programs decreased in dollar value from the current estimate. The average value of the 
six programs before the WSARA was $10.55 billion. The average value of the six 
programs after the WSARA was $6.01 billion, yielding a decrease of $4.54 billion. The 
average of all 17 programs before the WSARA was $10.35 billion. After the WSARA 
the average was $8.83 billion, yielding a net decrease of $1.51 billion. 

The 11 programs that increased only increased a very small amount. The 6 
programs that decreased nearly 3 times over the average decrease of the entire Army 
portfolio. However, this could only be an indicator or program cancellations, or 
appropriations funding being cut. Therefore, the dollar value is not a good indicator 
because of funding types and varying prices of programs within the portfolio. 

From before and after WSARA implementation, the standard deviation of nine 
programs increased from $297.38 million to $1,198.02 million. Eight programs’ standard 
deviations decreased from $1,572 million to $601 million. The combined standard 
deviation difference between programs before and after WSARA implementation 
indicated a $307 million net decrease. 

The standard deviation is a slightly better metric for interpreting current estimate 
changes. The programs that increased did so 4 times in magnitude, indicating that 
programs that did increase in standard deviation are becoming more spread out from the 
average. The programs that decreased did so by 2.6 times in magnitude, showing a 
slightly less dispersion around the mean. When comparing all programs, they indicate 
less dispersion but only a slight decrease. 

From before and after WSARA implementation, the CV of 10 programs increased 
from 2.76% to 16.40%. Seven programs’ CVs decreased from 22.5% to 8.25% after the 
WSARA was implemented; a decrease of 14.25%. The average of all programs’ CV 
went from 15.32% before to 13.04% after the WSARA, yielding a decrease of 2.27%. 
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As stated before, CV shows the dispersion of data relative to the mean beeause it 
allows for analysis of variation while eliminating the signifieant eost differenees and 
providing the answer in pereent form. The eurrent estimate overall deerease is not very 
signifieant. The program implieation is that eurrent estimates have beeome more in 
control by 2.27%. 

2. PAUC Percent Change 

PAUC is the program acquisition unit cost. It is computed by “dividing the 
Program Acquisition Cost by the Program Acquisition Quantity” (Hagan, 2012, p. B- 
177). It is basically how much everything costs divided by the amount of units that are 
(going to) be purchased. We compared the PAUC percent change from the original 
baseline estimate per each program SAR. 

From before and after WSARA implementation, five programs’ PAUC increased 
from 5.16% to 40.04%, a net increase of 34.88%. 12 programs’ PAUC percentage 

decreased from 4.72% to -2.73%, a net decrease of -7.45%. The combined average for 
all 17 programs’ PAUC percentage change against the baseline estimate increased from 
4.87% before the WSARA to 9.85% after, a net increase of 4.98%. 

From before and after WSARA implementation, the standard deviation of eight 
programs increased from 0.0418 before to 0.2221 after, a net increase of 0.1803. Nine 
programs decreased from 0.0953 before to 0.0357, a net decrease of 0.0596. The 
combined standard deviation average increased from 0.0856 to 0.1234, a net increase of 
0.0379. 

From before and after WSARA implementation, the CV of seven programs 
increased from 58.92% to 117.74%, an increase of 58.82%. Seven programs decreased 
from 226.34% to 95.22%, a decrease of 131.12%. Two programs did not have data to 
allow CV calculation before the WSARA. The combined average for 14 programs with 
available data to calculate CV yielded a 176.11% before to 106.48% after, a net decrease 
of 69.63. 
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The programs PAUC that did only increase did so by 34.88%. When compared to 
the standard deviation (.1803) of increasing programs, it a clear indicator of programs 
becoming more expensive. However, although this is highlighted with the CV of 
58.82%, the programs that decreased did so significantly by 131.12%. 

3. APUC Percent Change 

APUC is average procurement unit cost. It is “calculated by dividing total 
procurement cost by the number of articles to be procured” (that is, recurring and 
nonrecurring costs associated with production of the item (Hagan, 2012, p. B-19). It is 
basically the costs associated with physically making the product or item. We compared 
the APUC percent change from the original baseline estimate per each program SAR. 

From before and after WSARA implementation, five programs increased from 
8.69% to 14.79%, an increase of 6.1%. 12 programs decreased from 3.13% to 
6.02%, a decrease of 9.15%. The combined average change in APUC decreased from 
4.98% before to 0.10% after, a net decrease of 4.06%. 

From before and after WSARA implementation, 10 programs’ standard deviations 
increased from 0.0308 to 0.0979, an increase of 0.0671. Seven programs decreased from 
0.0963 to 0.0308, a decrease of 0.0655. The combined APUC standard deviation 
increased from 0.0578 to 0.0702, a net increase of 0.0125. 

From before and after WSARA implementation, the CV of 10 programs increased 
from 68.29% to 101.69%, an increase of 33.40%. Five programs decreased from 
124.99% to 39.65%, a decrease of 85.34%. The combined average CV decreased from 
99.79% to 81.01%, a net decrease of 18.78%. Two programs did not have data to allow 
CV calculation before the WSARA. 

Average procurement unit costs are decreasing on average. The standard 
deviation increase shows that they are becoming more difficult to control; however, the 
CV indicates that the APUC for the Army portfolio is decreasing. 
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E. INTENDED EFFECTS OF THE WSARA 

This section is based off the interviews we conducted. Each subsection of the 
WSARA’s Title II focused on changes to current or new acquisition policy. These 
questions were part of the interview process conducted at PEO CS&CSS to determine 
whether and how these legislative requirements are implemented within this organization. 
Interview questions were primarily based on sections of WSARA Title II. 

1. Consideration of Trade-offs for Requirement Development 

Section 201 of the WSARA states that “the Secretary of Defense shall ensure that 
mechanisms are developed and implemented to require consideration of trade-offs among 
cost, schedule, and performance objectives as part of the process for developing 
requirements for Department of Defense acquisition programs” with the duty of the 
Milestone Decision Authority to ensure that “the program is affordable” (Weapon System 
Acquisition Reform Act of 2009, 2009). 

Program requirements are generated at the Army G3/G4 level and disseminated 
to PEOs and PMs. PMs have little say on MDAPs to control trade-offs among cost, 
schedule, and performance objectives. However, the project managers did have influence 
on the MDA decision-making process on trade-offs as indicated by WSARA 201(f) 
DUTIES OE MIEESTONE DECISION AUTHORITY: 

Section 2366b(a)(I)(B) of such title is amended by inserting “appropriate 

trade-offs among cost, schedule, and performance objectives have been 

made to ensure that” before “the program is affordable.” 

Trade-offs had typically been made by the time the current project manager was 
selected and was actively managing a project. Objectives are typically already set in the 
program baseline CPD. If there are changes, project managers must provide rationale for 
the need to change their baseline estimates in cost, schedule, or performance. 

Certain programs had less stringent trade-off requirements. Eor example, the 
MRAP program could change baseline estimates to meet dynamic requirements driven by 
the Global War on Terror situations in Iraq and Afghanistan. As one PM put it, “Cost 
was third, schedule was second, and actually you balance it based on theater priorities.” 
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The Joint Light Tactical Vehicle (JLTV) was a cost informed trade driven platform. 
Tradeoff requirements were pushed to the contractor level based off of tiered capability 
development documents (CDDs) and KPPs. 

The spirit of the legislation matches what the PMs were doing with their 
respective projects, depending on state of the acquisition cycle. WSARA Section 
201(a)(2)(A-B) states that: 

(A) Department of Defense officials responsible for acquisition, budget, 
and cost estimating functions are provided an appropriate opportunity to 
develop estimates and raise cost and schedule matters before performance 
objectives are established for capabilities for which the Chair-man of the 
Joint Requirements Oversight Council is the validation authority; and (B) 
the process for developing requirements is structured to enable 
incremental, evolutionary, or spiral acquisition approaches, including the 
deferral of technologies that are not yet mature and capabilities that are 
likely to significantly increase costs or delay production until later 
increments or spirals. 

There was clear evidence that the JLTV program did exactly what WSARA 
201(c) required of review of joint military requirements. However, the older programs 
that are were already established did not meet this requirement. 

2. Competition Policy 

The WSARA mandates competition in as many phases of acquisition as possible. 
WSARA section 202 makes suggestions for competition during: 

(1) Competitive prototyping; 

(2) Dual-sourcing; 

(3) Unbundling of contracts; 

(4) Funding of next-generation prototype systems or subsystems; 

(5) Use of modular, open architectures to enable competition for upgrades; 

(6) Use of build-to-print approaches to enable production through multiple 
sources; 

(7) Acquisition of complete technical data packages; 

(8) Periodic competitions for subsystem upgrades; 

(9) Licensing of additional suppliers; 
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(10) Periodic system or program reviews to address long-term competitive 
effects of program decisions (Weapon System Acquisition Reform Act of 
2009, 2009). 

PEO CS&CSS competition policies were assessed to determine the strategies 
employed to ensure competition of products given the stage of the product in the 
acquisition life cycle. Reverse engineering techniques are the primary methods employed 
to create their own technical data package (TDP), which would drive more competition, 
because the contractors put a high cost on the procurement of TDPs. In addition, the PMs 
primarily used sole-sourcing due to priorities and the “finite amount of OEMs [original 
equipment manufacturers]” in the given industry or program. 

In contrast, the MRAP and JETV program focused on values trade-offs with 
performance-based competitions that helped industry partners be successful, as well as 
maintaining competition requirements. The need to develop the program requirements 
and have competition before Milestone A with defense acquisition executive (DAE) and 
component acquisition executive (CAE) support, would facilitate enforcement. 

3. Prototyping 

WSARA section 203 requires prototyping for MDAPs before Milestone B 
approval. PEO CS&CSS policy concerning prototyping, its frequency of use, potential 
costs, and the number of times prototyping is waived was analyzed to determine the level 
of compliance with this WSARA requirement. 

WSARA 203: (a) COMPETITIVE PROTOTYPING.—Not later than 90 
days after the date of the enactment of this Act, the Secretary of Defense 
shall modify the guidance of the Department of Defense relating to the 
operation of the acquisition system with respect to competitive 
prototyping for major defense acquisition programs 

Prototyping is required unless the requirement is waived by the MDA, prototypes 
exceed the expected life-cycle benefits, or DOD is unable to meet critical national 
security objectives. 

We assessed that prototyping does not typically happen in production phases. As 

most of the programs at PEO CS&CSS were in the later phases, the need to prototype 

was not as prevalent. However, there were some cost savings incentives to prototype if 
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modifications to systems were being made. In that case, the PMs typically used RDEC 
(Research, Development and Engineering Center) to develop a prototype or a TDP and 
then had contractors develop their solutions for the solicitation, thus improving 
competition. 

In contrast, rapid initiatives (like MRAP) did not have prototypes as per the 
WSARA requirement. The urgent needs of the Army drove the requirement, and they 
had companies produce units based off performance specs to get equipment out to the 
troops faster and leverage the waiver elements of the legislation. 

4. Systemic Problems and Product Termination 

Eor this section, our research efforts focused on the policies and procedures 
employed by PEO CS&CSS concerning identification of systemic problems and methods 
of determining when a product line must be terminated to determine if deviations from 
WSARA policy intent and application exist. 

The systemic problems were typically identified through internal reporting 
procedures at project level and presented to the MDA. If problems are identified, 
WSARA Title II sec 204 requires the MDA to certify costs, needs, program duplicates, 
relevancy, and that depot-level maintenance and repair capabilities have been made, and 
conduct an analysis of alternatives consistent with DCAPE guidance. 

WSARA sec 206 addresses product termination. The PMs reported changes to 
cost, performance, or schedule. If the changes were detrimental, the WSARA requires 
the PM to determine the root cause of cost growth or schedule threshold breach. The 
reports for these changes followed the same reporting processes as the systemic 
problems. However, it is the MDA that ultimately determines whether a product or 
project is continued or terminated. 

The reporting structure in PEO CS&CSS allowed for multiple reports on cost 
performance and schedule from product managers up to the project managers to the PEO. 
This facilitated the requirement from WSARA 204 with each product/project manager 
able to customize their reports to explain an issue at hand. Recommendations made at the 
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PM level with supporting cost, schedule, and performance data facilitated decision¬ 
making capabilities for the MDA on issues and termination if needed. 

5. Effectiveness of WSARA Policies 

WSARA policy relevance and cost effectiveness for PEO CS&CSS was 
addressed to determine whether there is added value from this policy within the PEO. 
Cost effectiveness is a measure of the results added by a system or in this case, legislative 
requirements (Hagan, 2012, p. B-49). We determined that the WSARA legislation 
requiring the policies that PEO CS&CSS followed were generally cost effective. 
However, there was some distaste for the reporting requirements; although deemed 
necessary, the reports may not provide significant value. 

F. WSARA IMPLEMENTATION 

Eor this section, we focused our inquiries on reporting requirements and personnel 
evaluation criteria within PEO CS&CSS. Studying these items enhanced our abilities to 
determine what is important to the PEO and whether or not WSARA policies are among 
the list of important items. We gauged their significance by level and frequency of 
reporting and whether or not WSARA policy items are included in personnel evaluation 
criteria. WSARA assists with comparing and evaluating the importance of these 
reporting policies within the PEO and whether or not they appear to be effective and 
efficient. 

I. Cost, Schedule, and Performance Reports 

The GAO study mentioned in the beginning of this project stated that incentives 
are partially to blame for inadequacies in acquisitions (Erancis, 2009). We wanted to 
determine how incentives influenced the reporting process. We learned that acquisition 
managers and leaders are incentivized to maximize fund obligation rates more than 
anything else. While there is a push to meet the needs of soldiers and that is typically 
where acquisitions begin, somewhere along the line, it becomes about obligating funds. 

In terms of what is reported, the research indicated a wide range of differences. 
Reform efforts (like the WSARA) stimulate additional reporting. Our researched indicated 

41 



that there are more reports required than can be numbered. The acquisition process 
becomes muddied when reform efforts create deviations from standard reporting 
requirements. Reporting has become such an arduous process at the PEO that personnel 
must be allocated solely to tracking reports and ensuring that reports are submitted on time. 
Much of this reporting appears to be required from those initiating the reform efforts so that 
these individuals have evidence that their reforms are creating change. 

Programs at PEO CS&CSS in earlier phases of the acquisition life cycle have 
fewer reporting requirements than those in the latter phases. Some of the common 
reports include: a weekly report sent to everyone in their product line, weekly staff calls, 
metrics and portfolio reviews, and Commander’s Critical Information Requirements 
(CCIR) reporting. 

We determined that PEO CS&CSS management trust that all of their reports go to 
a higher organization beyond their PEO, albeit a defense acquisition executive (DAE) if 
one exists. Army Budget Office, or Congress. In terms of which reports are more 
important than others, there appears to be a focus on obligation of funds and that these 
reports are prioritized with regard to what is submitted to higher tiers. 

There are not strict reporting structures. We determined that program cost 
reporting is stressed above all else when we studied PEO reporting hierarchies. 
Especially in times of fiscal constraints, reporting has become even more about cost. 
However, this is not reporting to show how costs are mitigated. Rather, this reporting 
focuses on obligation rates. There seems to be little concern with holding managers to 
items mentioned within WSARA Title II policies. The focus is on ensuring funds are 
obligated fully and on time. None mentioned having to report specifically on any aspect 
of the WSARA reforms. 

2. Reporting Management 

Report oversight is another factor used in our research to indicate the importance 
of what is reported. Those items with greatest oversight are determined to be most 
important to the PEO and their superiors. Our analysis for this section focused on PEO 
CS&CSS reporting processes, including the number of reports, and the level of oversight 
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or senior management involvement. The level of oversight varied based on where a 
respective program fell within the acquisition life cycle. Those programs still in the early 
phases of the acquisition life cycle stated that they feel there is little oversight to what 
they do on a regular basis. However, managers of programs operating in the latter phases 
of the acquisition life cycle stated they are “over sighted to death.” 

While oversight varied depending on acquisition life cycle phase, there was a 
consensus that reports were not actually reviewed beyond the PEO level based on the 
absence of feedback. Often, feedback is received concerning reporting only if there is a 
combat-related requirement. For instance, if a soldier is wounded or killed in combat 
while employing a product in their program, this generates changes to an aspect of their 
product (armor for example), and oversight becomes abundant. Otherwise, for programs 
in early acquisition phases, there seems to be little oversight to what is going on in a 
program. When a problem presents itself in a program is another instance of when report 
feedback is prominent. Specifically, reports concerning funding problems receive an 
abundance of feedback along with additional oversight indicating that funding and cost 
are valued above all other report items. 

3. Performance Assessments and the WSARA 

Performance evaluations and metrics are a method employed by supervisors to 
incentivize behavior in congruence with organizational goals. Investigating whether 
personnel evaluations, within PEO CS&CSS, included requirements from the WSARA 
demonstrates the level of the WSARA policies significance to the PEO. We focused our 
research on specific WSARA aspects including reporting, cost control, and requirements 
development, and analyzed whether or not these policies were evaluated in either the 
civilian or military performance evaluations. We discovered that personnel are not 
evaluated on (nor evaluate their subordinates on) anything directly stemming from 
WSARA policies. Considering adherence to policy initiatives when conducting 
evaluations is something worth implementing for managers and those under their 
evaluation. Evaluations at PEO CS&CSS are more about percentage of funds obligated 
or number of upgrades installed, for example, than they are about managing cost, 
schedule, and performance. 
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There is discontinuity between civilian and military personnel evaluations as well 
according to those interviewed. Civilian evaluations are very vague and do not tell much 
about their performance unless they did something wrong. The evaluation process is 
further complicated due to lacking longevity of military personnel within a program 
office. As such, evaluations become more about what kind of person the military 
individual is, rather than how they manage a program with regard to items such as those 
mentioned in the WSARA policies. Because that individual is not in the program from 
conception through final phases of product acquisition, it is difficult to measure their 
cost, performance, and schedule contributions. 

4. Reporting Effectiveness 

We assessed the effectiveness of the PEO CS&CSS reporting process and 
contents of submitted reports to identify whether inefficiencies exist and ascertain the 
implications of continued acquisition reforms (such as the WSARA) on this process. We 
established that the current reporting process is not effective. The consensus was that the 
acquisition process itself is very effective. However, everything that is continually added 
on top of this process, such as WSARA reforms and the reporting requirements that come 
with reform, take away from the process’s effectiveness. There are opportunity costs 
associated with having to obligate personnel in a program to managing nothing but 
reporting requirements. This takes away from their abilities to add real value to other 
aspects within a program. 

Additionally, there is little feedback with regard to the reports that they submit on 
a regular basis. New policies often create new reporting requirements, but acquisition 
managers are not confident that these reports amount to much more than simply 
satisfying a requirement to report. Policy reforms that lead to additional reporting 
requirements appear to be nothing more than another person’s (the person implementing 
the reform) attempt to have their part in history. This leads to ineffective reporting due to 
the time it takes to generate these reports and the tasks that may fall by the wayside 
because time was spent reporting rather than managing cost, schedule, performance and 
risk. 
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Initial reporting requirements incumbent to the baseline acquisition process are 
adequate. An acquisition reform such as the WSARA, adds to the reporting 
requirements. As more reform initiatives are made, more reporting is required. Further, 
no unnecessary or outdated reporting requirements are eliminated. These new and old 
reporting requirements compound on top of each other, until only reporting is happening. 
The original intent of the reporting is lost, and no one has any idea why the reporting is 
being done and whether or not reports are necessary at higher levels anymore. 

G. POLICY IMPLEMENTATION AND GAPS 

This section presents information demonstrating whether there was deviation 
from the WSARA’s intent and application within PEO CS&CSS and if so, at what level, 
and what is actually implemented within programs. 

I. Tracking Acquisition Policy 

Acquisition policy and reform is constantly being reviewed and updated. This 
process is managed through policies at all levels of the bureaucracy. Acquisition 
managers within PEO CS&CSS have challenges tracking changes to policy. Some policy 
changes or updates to procedure are relatively simple to print and make a cheat sheet to 
check your program efforts against. However, complex policy reforms and legislative 
changes such as the WSARA are more difficult, and this difficulty is compounded when 
one considers that the WSARA is only one of many policy documents requiring 
adherence. 

Additionally, program managers are not pressured to continually check their 
programs against policy. Resources such as the Defense Acquisition Portal (DAP) and 
Defense Acquisition University (DAU) were cited as potential sources to consult when 
searching for policy documents when evaluating programs. But given the amount of 
policy and reforms that exist, this can be a lengthy process. 

True policy reviews come when the program is reaching certain milestone or 
decision briefings within the acquisition process. At these points, the policy making 
personnel scrub documents and procedures against policy requirements, which assists 
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with ensuring the program is in compliance. However, due to the ever increasing amount 
of change and reforms that takes place with the acquisition process, the time it takes to 
process these documents and briefings is ever increasing. 

2. Policy Challenges 

The requirements created by the WSARA created bureaucratic challenges. One 
of the programs mentioned having difficulties implementing competition throughout the 
life cycle of their program due to TDP issues. The program acknowledges that full and 
open competition is a requirement; however, if one contractor holds the TDP for a piece 
of equipment or system, competition is compromised. Purchase of the TDPs from the 
contractor is often impossible in circumstances where the contractor put the TDPs out of 
reach from the government by placing such a high cost on the TDPs that it is outside the 
program’s funding capabilities. 

Policy reforms and changes are difficult to implement based on how cumbersome 
the process becomes. When a policy is reformed, it must be implemented. Then 
personnel must be employed to check policies and generate a database for ensuring a 
program is evaluated against the policy efforts. Additional personnel are then deviated 
from their prior duties to start reporting and ensuring that the database and reporting is 
feeding the individuals who require the reporting. Frustration with this process abounds 
as it shows that reporting compliance with policy is more significant than actually 
ensuring correct policy implementation and sound program management. 

The major roadblock programs have encountered with implementing the 
competition policy from the WSARA is the TDP issue. Also, the cumbersome process of 
tracking, creating metrics to measure a program against, reporting, and tracking of 
reporting requirements that are created with each additional change to the acquisition 
process are disincentives to policy implementation. 

3. Policy Successes 

The requirements created by the WSARA have created opportunity for success. 
Being forced to consider trade-offs between cost, schedule, and performance in the early 
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years of a program changes acquisition approaches. Successes have occurred through 
utilization of the DODI 5000 process guides and other instances persist where individuals 
within their programs utilized various policies to produce rapid results when a combat 
need was identified. 

Positive feedback in programs was not directly linked to WSARA policy. Most 
positive things that happen in programs result from following acquisition policy and 
endeavoring to meet the needs of end users. However, the correlation between these 
positive outcomes and implementation of a WSARA Title II policy is not easily 
delineated. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. INTRODUCTION 

The final section of this project addresses our research conclusions and 
recommendations. These items are grouped within the categories of quantitative, 
organizational and procedural, Title II WSARA, and summary conclusions and 
recommendations. We initially address statistical conclusions drawn from the DAMIR 
analysis. The remaining conclusions and recommendations are based primarily on 
interview trends as well as any other research findings. 

I. Conclusions Summary 

There is no direct, measurable linkage between Title II WSARA policies and 
improvements to processes, organizational structure, costs, schedule, and 
performance. Priorities and incentives drive the acquisition process. Within the 
acquisition process and as new acquisition officers, we are constantly counseled to 
consider cost, schedule, and performance within acquisition programs. Indeed, the 
research indicated that these three items, along with risk, as equally important. However, 
as interviews were conducted, discrepancies amongst how these items are prioritized 
within a program surfaced. 

Managers and workers of programs respond to incentives. Incentives present 
themselves in the form of career advancement, excellent evaluations, praise, and many 
others. When and why these incentives are delivered to a recipient establish the 
program’s priorities. So, while a manager may say cost, schedule, performance, and risk 
are equally important, their actions in only rewarding when funds are fully obligated lead 
subordinates to determine that the true priority is money obligated and spent in a program 
vice what were the stated priorities. 

Adding to the already numerous policy documents that exist to govern the 

acquisition process is not the most effective or efficient manner to improve acquisitions. 

Rather, enforcing requirements for PdMs and other representatives to adhere to the 

acquisition process while establishing priorities at the PEO and below level (priorities 
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determined from evolving needs and fiscal requirements) and incentivizing all involved 
to adhere to these priorities will deliver desired improvements. 

B. CONCLUSIONS 
1. Quantitative 

It is clear that MDAP cost variability is in more control on current estimate and 
PAUC and APUC baseline comparisons since the implementation of the WSARA. 

As stated in the data analysis section, we focused on analyzing the measures of 
central tendency for the selected elements of cost in the current estimate, PAUC change 
against baseline, and APUC change against baseline from each Army MDAP SARs that 
were available from 2005 to 2012. 

The reader’s focus should shift to the coefficient of variation (CV), as this 
measurement equalizes the playing field because it is has no unit, nullifying actual dollar 
figures or program baseline estimates, and compares the program standard deviation 
divided by its average, giving relative magnitude and relative variability. For example, 
from Jeff Sauro’s website: 

if the mean is 80 and standard deviation is 12, the cv = 12/80 = .15 or 
15%. If the standard deviation is .20 and the mean is .50, then the cv = 

.20/.50 = .4 or 40%. So knowing nothing else about the data, the CV helps 
us see that even a lower standard deviation doesn’t mean less variable 
data. 

We calculated the differences of coefficient of variation (CV) between the 
samples (for each SAR cost element) before the WSARA and then after WSARA 
implementation. 

For our example, reference Table 1 of this project. All of the MDAPs’ APUC CV 
before the WSARA was 99.01% and after was 81.01%, showing the change of -18.78% 
in Table 2 below. For the programs that only decreased in CV, they did so by -85.34%, 
shown in Table 2 (with the data computation drawn from Table 1). Each of the values in 
Table 2 that decreased indicates a cost measurement that is more in control (and less in 
control if the CV increased). 
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Since WSARA implementation, the sampled programs have improved in cost 
control. The spirit of the law intended to have more control on programs and lower 
costs. 

Table 2 summarizes the changes on programs that increased or decreased before 
the implementation of the WSARA and after. In addition, there is a combined net change 
in each measurement showing the total change. 


Table 2. Summary SAR MDAP Changes In Central Tendency 


SAR Report Data _ Average _ Stdev _ CV 

Current Estimate $160.09 $900.64 13.64% 

PAUC% Change 34.88% 0.1803 58.82% 

APUC% Change 6.10% 0.0671 33.40% 

PROGRAMS THAT DECREASED 


SAR Report Data _ Average _ Stdev _ CV 

Current Estimate $(4,538.13) $(970.56) -14.24% 

PAUC % Change -7.45% -0.0596 -131.12% 

APUC% Change -9.15% -0.0655 -85.34% 

COMBINED NET CHANGE FOR ALL PROGRAMS 


SAR Report Data 

Average 

Stdev 

CV 

Current Estimate 

$(1,514.21) 

$(306.97) 

-2.21% 

PAUC % Change 

4.98% 

0.0379 

-69.63% 

APUC % Change 

-4.06% 

0.0125 

-18.78% 


It is clear that MDAP cost variability is in more control on current estimate and 
PAUC and APUC baseline comparisons since the implementation of the WSARA. 
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2. Organizational and Procedural Impacts 

Our research indicated no true change to organizational structure, task allocation, 
or administrative functions directly linked to WSARA implementation. The 
organizational structure at the PEO level and above, is very structured. This is necessary 
to establish clear linkages and lines of responsibility from PEO to congressional leaders, 
DAEs, and others. However, at the PdM and below level, the organizational structure is 
highly fluid and was different within each program that we interviewed, as were their 
procedures. The structure was tailored to focus personnel on the priorities of the 
program. Some PdMs managed people and tasks by functional category (logistics, 
operations, administration, engineering etc.) while others were organized by product line 
or subcategory within the program itself. 

This decentralized approach to managing structure and procedures at the PdM 
level and below is effective and efficient according to program leaders. While none 
interviewed stated the WSARA as a reason for changing organizational structure and 
procedures, it is possible that, due to influence from representatives at the PEO level and 
above whose actions may have been motivated by the WSARA, the reform is partially 
responsible for some organizational changes. 

The research suggests that a negative organizational and procedural impact 
resulting from acquisition reforms and changes to policy. This negative impact is seen in 
the increased amount of reporting and tracking or reporting that is required each time new 
policy is implemented. Increased reporting leads to allocation of resources (personnel 
and time) to reporting requirements, decreasing resources that have previously been 
focused on other important management tasks. 

3. WSARA Title II Policy Application Conclusions 

Title II WSARA policies focus on evaluation of tradeoffs between cost, performance 
and schedule, prototyping, maintaining competition throughout a program’s life cycle, 
addressing systemic problems, and terminating product lines. Throughout our initial research 
and program representative interviews, we determined that the acquisition process and 
policies already have procedures and rules concerning what Title II WSARA requires. 
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Additionally, all programs do not fall in the same space and time within the 
acquisition process. So, when a new requirement to ensure competition throughout a 
product’s life cycle is implemented in a program in its latter (Operations and Support) 
phase of the acquisition process, this may lead to schedule delays. The research 
indicated that, in such programs, the TDPs are often out of their reach. New competition 
requirements likely require waivers if competition cannot take place, which may result in 
schedule overruns due to processing delays. This is one example of many possible 
scenarios where policy changes lead to delays or other tradeoffs within programs. 

C. RECOMMENDATIONS 

The quantitative findings indicate programs that are lower and more in control on 
average since the implementation of WSARA. However, a larger sample size of cost 
data from SARs or other sources would increase statistical significance. Although there 
is not direct causation identified, there is a clear correlation of the WSARA’s impacts on 
Army MDAPs. We recommend continuing to implement WSARA policy requirements 
and lessen other acquisition reform to control for the WSARA cause and effect. 

To say that each program is the same and thereby should maintain a pre¬ 
designated organizational structure at the PdM level and below would force programs to 
reorient themselves in a manner that is likely to be less capable and effective than they 
are currently. PdMs must continually evaluate their organizational structure and 
procedures in order to determine how to best tailor their programs to meet the 
requirements in accordance with where their program falls in the acquisition process and 
priorities established by their PEO and above. 

PEO and above representatives should be responsible for establishing procedural 
and organizational priorities that are in line with policies and reforms. People respond to 
incentives and react according to that which is rewarded. When PEOs establish priorities 
and evaluate based on these priorities it ensures PdMs focus on those priorities without 
needing to know each change to policy that occurs. 

Our research found that policy-makers may not be evaluating whether or not a 
policy reform is required prior to implementing a change. In other words, policy changes 
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become more about a policy-maker wanting to make his or her mark on history while 
responding to pressures from external forces such as public criticism to fix military 
acquisitions. Thus, a change becomes implemented which requires reporting, waivers 
and tracking or all of these items, resulting in a potential detriment to effective program 
management. 

Policy-makers must evaluate the baseline acquisition process and its supporting 
policy by ensuring that reforms are not made when they are not needed. Further, reforms 
need not be repetitive in nature. When repetitive changes are made to policies, this is an 
indicator that incentives and priorities are mismatched. A new reform is not necessary. 
Rather, PEOs and PdMs need to understand what their priorities are and be managed 
through incentives to adhere to those priorities. 
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